Design Science Research Post Hevner et al.: Criteria, Standards, Guidelines, and Expectations

There is ongoing debate about how the quality (rigour and relevance) of Design Science Research (DSR) should be judged. This research investigates the state of the debate by surveying the opinions of IS scholars who write, review, edit, and publish DSR papers. The survey respondents rated the relative importance of the seven guidelines (often used as evaluation criteria) laid out in Hevner et al (2004), more specific criteria about the evaluation activity in DSR, criteria concerning IS Design Theories, and miscellaneous other criteria, and made general open-ended comments. The findings indicate a lack of consensus, with much variability in ratings. The Hevner et al (2004) guidelines are largely endorsed, but caution is also raised to apply them less mechanistically than at present. Some criteria/guidelines are seen to be less important at earlier stages of research. Caution is also urged not to expect single papers to fit all criteria/guidelines.